Skip to content
This repository was archived by the owner on Jun 12, 2023. It is now read-only.

Build Wheels with GH Actions #29

Closed
wants to merge 27 commits into from
Closed

Build Wheels with GH Actions #29

wants to merge 27 commits into from

Conversation

WillAyd
Copy link
Member

@WillAyd WillAyd commented Feb 12, 2020

WIP to see how this might look

@jbrockmendel
Copy link
Member

Is this based on a template taken from somewhere?

@WillAyd
Copy link
Member Author

WillAyd commented Feb 12, 2020

The actions are from another project I worked on with similar build requirements. The docker parts for the manylinux build come from the pypa demo project

https://github.com/pypa/python-manylinux-demo

You can see results of this in my own repo (in progress)

https://github.com/WillAyd/pandas-release/pull/1/checks?check_run_id=441681755

@WillAyd
Copy link
Member Author

WillAyd commented Feb 12, 2020

OK got a minimum example working on GitHub actions. Here is what it looks like:

https://github.com/WillAyd/pandas-release/pull/1/checks?check_run_id=441916246

image

The artifacts are really zip files containing individual wheels. We could alternately put all of the wheels into one zip file

@WillAyd
Copy link
Member Author

WillAyd commented Feb 12, 2020

A few things worth discussing:

  1. GH Actions by default only has a Windows environment to build a win_amd64 wheel. We also produce win32 wheels today, which I think are 32 bit Windows wheels (not sure how Windows names things). If that is correct, would need another way to build 32 bit Windows (maybe Docker)
  2. The macOS wheels come with platform tag macosx_10_13_x86_64 (the actual environment is 10.15). We offer 10_9 on PyPi, so if we want to go that far back again need another option. Note that 10_13 came out in Sep 2017 and 10_9 came out in Oct 2013
  3. I didn't add a make rule to download / unpack the wheels, though I think we would need that

One other thing I had but remove from the process was installing and running pandas.test() as part of the action. This seemed to fail by default on macOS / Windows and the installation for the ubuntu images are slightly more complex, so didn't bother with them for now

Would love any feedback anyone has to offer

@jreback
Copy link

jreback commented Feb 13, 2020

we wouldn’t need to actually run .test() because by definition we only would build these after the. CI is green (it would be nice to make sure that the wheels actually install properly)

we should just drop win32 entirely

@WillAyd
Copy link
Member Author

WillAyd commented Feb 13, 2020

Updated to drop 32 bit wheel generation. I also added a step to install and run import pandas with the wheel to be sure

python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install setuptools wheel cython numpy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's needed to install dependencies, the wheel building step should create an isolated environment with the dependencies listed in pyproject.toml?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, maybe I am mistaken and pyproject.toml is only used when using pip commands (like "pip install" when building from source). Sorry about that.

In any case, if installing numpy, you need to install the oldest supported version to have compatible wheels for different numpy versions.

@jorisvandenbossche
Copy link
Member

My main concern is whether we want to start using something custom, versus tooling that is also used (and thus partly maintained) by other packages in the ecosystem (multibuild is now used by many like numpy, scipy, scikit-learn, ...)

@WillAyd
Copy link
Member Author

WillAyd commented Feb 13, 2020

Its a good question. I just think this is much easier than trying to configure multi build and keeps everything in Azure

@TomAugspurger
Copy link
Collaborator

FYI: MacPython/scikit-learn-wheels#23 (building on azure-pipelines)

@WillAyd WillAyd closed this Mar 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants